Since personal computers started to be spread in 1980's, technology, performance and environment of computers have been rapidly developed. In 1990's, the Internet was rapidly applied to various fields of companies and personal lives. Therefore, computers are going to be very important in every field throughout the world in the 21st century. One of the computer music applications is musical instrument digital interface (MIDI). MIDI is a representative computer music technique used by musicians to synthesize and/or store musical sounds of instruments or voices. At present, MIDI is a technique mainly used by popular music composers or players.
For example, composers can easily compose music using computers connected to electronic MIDI instruments, and computers or synthesizers can easily reproduce the composed MIDI music. In addition, sounds produced using MIDI equipments can be mixed with vocals in studios to be recreated as a popular song having support of the public.
The MIDI technique has been developed in combination with popular music and has been entered to musical education field. In other words, MIDI uses only simple musical-information like instrument-types, notes, notes'-strength, onset and offset of notes regardless of the actual sounds of musical performance so that MIDI data can be easily exchanged between MIDI instruments and computers. Accordingly, the MIDI data generated by electronic-MIDI-pianos can be utilized in musical education using computers, which are connected to those electronic-MIDI-pianos. Therefore, many companies including Yamaha in Japan develop musical education software using MIDI.
However, the MIDI technique does not satisfy the desires of most classical musicians treasuring sounds of acoustic instruments and feelings arising when playing acoustic instruments. Because most of the classical musicians do not like the sounds and feelings of electronic instruments, they study music through traditional methods and learn how to play acoustic instruments. Accordingly, music teachers and students teach and learn classical music in academies of music or schools of music, and there is no other way for students but to fully depend on music teachers. In this situation, it is desired to apply computer technology and digital signal processing technology to the field of classical music education so that the music performed on acoustic instruments can be analyzed and the result of analysis can be expressed by quantitative performance information.
For this, digital sound analysis technology, which digital sounds are converted from the performing sounds on acoustic instruments, has been developed using computers in various viewpoints.
For example, the method of using score information to extract MIDI data from recorded digital sounds is disclosed in a master's thesis entitled “Extracting Expressive Performance Information from Recorded Music,” written by Eric D. Scheirer. This thesis relates to extracting of the notes'-strength, onset timing, offset timing of each note and converting the extracted information into MIDI data. However, referring to the results of experiments described in the thesis, onset timings were accurately extracted from recorded digital sounds to some extent, but extraction of offset timings and notes'-strength of notes were inaccurate.
Meanwhile, several small companies in the world have put initial products that can analyze simple digital sounds using a music recognition technique on the market. According to the official alt.music.midi newsgroup FAQ (frequently asked questions), which is on the Internet page http://home.sc.rr.com/cosmogony/ammfaq.html, there are some products to convert wave files into MIDI data or score data by analyzing the digital sounds in wave files. The products include Akoff Music Composer, Sound2MIDI, Gama, WIDI, Digital Ear, WAV2MID, Polyaxe Driver, WAV2MIDI, IntelliScore, PFS-System, Hanauta Musician, Audio to MIDI, AmazingMIDI, Capella-Audio, AutoScore, and most recently published WaveGoodbye.
Some of these products are advertised as being able to analyze polyphonic-sounds. However, it was found that they could not analyze polyphonic-sounds as a result of experiments. For this reason, the FAQ document describes that the reproduced MIDI sounds cannot be heard just like the original sounds after the sounds have been converted into MIDI format. Moreover, the FAQ document plainly states that all software published at present for converting wave files into MIDI files are of no worth.
The following description concerns the result of the experiment on AmazingMIDI by Araki Software to find how it analyzes polyphonic-sounds in a wave file.
FIG. 1 is a piece of musical score used in the experiment and shows first two measures of the second movement in Beethoven's Piano Sonata No. 8. In FIG. 2, the score is divided in units of monophonic notes for convenience of analysis, and the note names are assigned to the individual notes. FIG. 3 shows a parameter setting window on which a user sets parameters for converting a wave file into a MIDI file in AmazingMIDI. FIG. 4 is a window showing the converted MIDI data obtained when all parameter control bars are fixed at the right-most ends of control sections. FIG. 5 shows the expected original notes based on the score of FIG. 2 using black bars on the MIDI window of FIG. 4. FIG. 6 is another MIDI window showing the converted MIDI data obtained when all the parameter control bars are fixed at the left-most ends of the control sections. FIG. 7 shows the expected original notes using black bars on the MIDI window of FIG. 6, like FIG. 5.
Referring to FIGS. 1 and 2, three notes C4, A3♭, and A2♭ initially start. Then, in a state where piano keys corresponding to the notes C4 and A2♭ are pressed, keys corresponding to notes E3♭, A3♭, and E3♭ are sequentially pressed. Next, a note B3♭ follows the note C4, and simultaneously, notes D3♭ and G3 follows the notes A2♭ and E3♭, respectively. Then, in a sate where keys corresponding to the notes B3♭ and D3♭ are pressed, keys corresponding to notes E3♭, G3, and E3♭ are sequentially pressed. Accordingly, when this wave file based on the score is converted to MIDI data, MIDI data must be configured as expressed by black bars shown in FIG. 5. However, in the real experiment, MIDI data was configured as shown in FIG. 4.
Referring to FIG. 3, AmazingMIDI allows a user to set various parameters for converting wave files into MIDI files. Configuration of the MIDI data varied with the set values of these parameters very much. When the values of Minimum Analysis, Minimum Relative, and Minimum Note were set to the right-most values on the parameter input window of FIG. 3, MIDI data resulting from conversion was obtained as shown in FIG. 4. When these values were set to the left-most values, MIDI data resulting from conversion was obtained as shown in FIG. 6. When FIG. 4 is compared with FIG. 6, it can be seen that there is a lot of difference between them. In other words, only frequencies having large magnitudes in a frequency domain were recognized and expressed in the form of MIDI in FIG. 4, but frequencies having small magnitudes were recognized and expressed in the form of MIDI in FIG. 6. Accordingly, MIDI data shown in FIG. 6 basically contains MIDI data of FIG. 4.
When compared with FIG. 5, FIG. 4 shows that the notes A2♭, E3♭, G3, and D3♭ were not recognized at all, and recognition of the notes C4, A3♭, and B3♭ was very different from actual performance based on the score of FIG. 2. In detail, in the case of the note C4, recognized length is only initial 25% of original length. In the case of the note B3♭, recognized length is less than 20% of original length. In the case of the note A3♭, recognized length is only 35% of original length. Moreover, many notes that were not performed were recognized. A note E4♭ was recognized with loud notes'-strength, and unperformed notes A4♭, G4, B4♭, D5, and F5 were wrongly recognized.
When compared with FIG. 7, FIG. 6 shows that although the notes A2♭, E3♭, G3, D3♭, C4, A3♭, and B3♭ that were actually performed were all recognized, recognized notes were very different from the performed notes. In other words, the actual sounds of the notes C4 and A2♭ were continued since the keys were maintained pressed, but the notes C4 and A2♭ were recognized as being stopped at least one time. In the case of the notes A3♭ and E3♭, recognized onset timings and note lengths were very different from actually performed ones. In FIGS. 6 and 7, many gray bars show in addition to black bars. The gray bars indicate notes that were wrongly recognized although they were not actually performed. These wrongly recognized gray bars are more than correctly recognized bars. Although the results of experiments on programs other than AmazingMIDI program will not be described in this specification, it was proved that the results of experiments on all published programs for recognizing music were similar to the result of the experiment on AmazingMIDI program and were not satisfactory.
Although techniques of analyzing music performed on acoustic instruments using computer technology and digital signal processing technology have been developed in various viewpoints, satisfactory results have never been obtained.